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DETAILED ACTION 

Claim Objections 

1 . Claim 4 is objected to because it is not grammatically correct. Specifically, 
"whether to display the list image to switch whether to contain the list image in the 
virtual space image" is not clear, nor grammatically correct, although the Examiner can 
understand what was meant. Appropriate correction is required. 

2. Claims 14 and 15 are objected to because of the following informality: each 
structural limitation should begin with an "a" - for example, "first acquisition unit" should 
be amended to read -a first acquisition unit--. Appropriate correction is required. 

Claim Rejections - 35 USC §112 

3. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

4. Claims 1, 2 and 14 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. 

Regarding claims 1 and 14, the phrase "laying out a list image to display a list of 
pieces of information" is unclear. The term "laying out" has a very wide variety of 
inferences, both physical and graphical, and it is unclear what the scope of this 
limitation was intended to be. An example of more clear language might be, a layout 
step of generating a list image comprising a list of pieces of information. 

Regarding claim 2, this claim is rejected for similar reasons discussed above. 
Additionally, the phrase "making a plane of the list image" is also unclear. 
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5. Claim 3 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. Regarding this claim, the term "divides a line segment... to t : 
(1-t)" is unclear because the formula does not make sense in the scope of the claim, nor 
is "t" defined in the claims. 

Claim Rejections - 35 USC § 102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

7. Claims 1-4, 7, 8, and 11-17 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Kato et al. (U.S. Patent No. 5,999,185), referred herein as Kato. 

Regarding claim 1 , Kato teaches an image processing method of generating an 
image of a virtual space formed from a virtual object including at least one part 
(abstract), characterized by comprising: a first acquisition step of acquiring a position 
and orientation of a viewpoint of an observer (column 2, lines 38-50; column 17, lines 1- 
2); a second acquisition step of acquiring a position and orientation of a pointing device 
which is worn by the observer on a hand to execute various kinds of operations (figure 
1, pointing device 3A; column 10, lines 53-55; column 17, lines 1-7); a layout step of 
laying out a list image to display a list of pieces of information about the parts near the 
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position acquired in the second acquisition step (figure 18; figure 24; figures 24A, B, and 
C; column 6, lines 50-52; column 20, lines 27-44; column 21, lines 45-47 and 53-56; 
column 22, lines 62-66; column 23, lines 2-4, 18-23, 31-34, and 47-50); a virtual space 
image generation step of generating the image of the virtual space after laying out the 
list image, which is seen in accordance with the position and orientation of the 
viewpoint; and an output step of outputting the image generated in the virtual space 
image generation step to a predetermined display device (column 15, lines 58-62; 
column 17, lines 11-13 and 26-28). 

Regarding claim 2, Kato teaches the method according to claim 1, and further 
teaches the method, characterized in that in the layout step, the list image is laid out 
while making a plane of the list image, on which the pieces of information about the 
parts are described, visible from the position and orientation of the viewpoint (figure 18; 
figures 24A, B, and C; column 20, lines 27-44; column 23, lines 18-23). 

Regarding claim 3, Kato teaches the method according to claim 1 , and further 
teaches the method, characterized in that in the layout step, the list image is laid out at 
a position that internally divides a line segment connecting the position acquired in the 
first acquisition step and the position acquired in the second acquisition step to t : (1-t) 
(column 20, lines 28-32 and 47-51; column 36, lines 35-49). 

Regarding claim 4, Kato teaches the method according to claim 1 , and further 
teaches the method, characterized by further comprising a layout control step of 
controlling processing in the layout step in accordance with an instruction whether to 
display the list image to switch whether to contain the list image in the virtual space 
image (column 23, lines 2-5 and 9-22). 
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Regarding claim 7, Kato teaches the method according to claim 1 , and further 
teaches the method, characterized by further comprising a distance calculation step of 
calculating a distance between the position of the pointing device and the position of the 
virtual object (figures 35 and 36; column 29, lines 3-17), and a list image generation 
step of generating the list image to display the list of pieces of information about the 
parts up to a layer level corresponding to the distance calculated in the distance 
calculation step in a hierarchical structure of the parts included in the virtual object 
(figure 6; figures 48A, B, and C; column 11, lines 58-67; column 12, lines 1-7; column 
35, lines 23-46). 

Regarding claim 8, Kato teaches the method according to claim 1 , and further 
teaches the method, characterized in that the list image is an image to display a list of 
pieces of information about, of the parts included in the virtual object, a part at a position 
closest to the position of the pointing device acquired in the second acquisition step 
(column 1 7, lines 1 -7 and 11-13; column 20, lines 27-44; column 21 , lines 45-47 and 53- 
56; column 22, lines 62-66; column 23, lines 2-4, 18-23, 31-34, and 47-50). 

Regarding claim 1 1 , Kato teaches an image processing method of generating a 
virtual image corresponding to a position and orientation of an observer and 
compositing the virtual image with a physical image corresponding to the position and 
orientation of the observer, including acquiring the position and orientation of the 
observer, generating the virtual image in accordance with the position and orientation of 
the observer (column 2, lines 38-50; column 17, lines 1-2; column 28, lines 59-64), and 
compositing the physical image corresponding to the position and orientation of the 
observer with the virtual image (column 28, lines 59-64; column 29, lines 22-26), 
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characterized by comprising: acquiring a position of pointing means operated by the 
observer; identifying a part pointed by the pointing means (figure 1 , pointing device 3A; 
column 10, lines 53-55; column 17, lines 1-7); generating a part information virtual 
image to display information about the identified part; and determining a layout position 
of the part information virtual image in accordance with the position of the pointing 
means (figure 18; figure 24; figures 24A, B, and C; column 6, lines 50-52; column 20, 
lines 27-44; column 21, lines 45-47 and 53-56; column 22, lines 62-66; column 23, lines 
2-4, 18-23, 31-34, and 47-50). 

Regarding claim 12, Kato teaches the method according to claim 1 1 , and further 
teaches the method, characterized in that the layout position is determined in 
accordance with the position of the pointing means and the position and orientation of 
the observer (column 1 7, lines 11-13; column 20, lines 27-44; column 21 , lines 45-47 
and 53-56; column 22, lines 62-66; column 23, lines 2-4, 18-23, 31-34, and 47-50). 

Regarding claim 13, Kato teaches the method according to claim 1 1 , and further 
teaches the method, characterized in that the information about the part comprises a list 
of a plurality of parts associated with the identified part (figures 24A, B, and C; column 
21, lines 45-47 and 53-56; column 22, lines 62-66; column 23, lines 2-4, 18-23, 31-34, 
and 47-50). 

Regarding claims 14 and 15, the limitations of these claims correspond to the 
limitations of claims 1 and 1 1 , respectively; thus, they are rejected on the same grounds 
as the limitations of claims 1 and 1 1 , respectively. 

Regarding claims 16 and 17, Kato teaches a program characterized by causing a 
computer to execute an image processing method of claims 1 and 1 1 (figure 3, main 
program 18; column 16, lines 50-51). 
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Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

9. Claims 5 and 6 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Kato, in view of Buxton et al. (U.S. Patent No. 6,1 18,427), referred herein as Buxton. 

Regarding claim 5, Kato teaches the method according to claim 1 , but does not 
teach the method, characterized by further comprising a conversion step of converting 
the list image into a semitransparent image in accordance with an instruction to change 
the list image to the semitransparent image. Buxton teaches a method for optimal 
transparency processing in a graphical user interface to merge graphical images of 
objects, comprising conversion of the images into semitransparent images in 
accordance with an instruction to change the images to the semitransparent image 
(column 3, lines 40-46). As was very well known to those of ordinary skill in the art, and 
illustrated by Buxton, image transparency enables a user to view multiple layers of 
information without losing information that would otherwise be occluded by overlapping 
images, thus increasing the efficiency and simplicity of the user interaction, as well as 
improving the quality of the imaging system as a whole. Therefore, it would have been 
obvious to a person having ordinary skill in the art at the time the invention was made to 
combine the semitransparent images of Buxton with the invention disclosed by Kato. 
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Regarding claim 6, Kato teaches the method according to claim 1 , and further 
teaches the method, characterized by further comprising a determination step of 
determining on the basis of the position and orientation of the viewpoint and a position 
of the virtual object whether the virtual object is present in a direction of line of sight of 
the viewpoint (column 17, lines 1-7; column 40, lines 55-63). Kato does not teach the 
method, further comprising a transparency control step of, when it is determined in the 
determination step that the virtual object is present, making a transparency of the list 
image higher than that when it is determined in the determination step that the virtual 
object is not present. Buxton teaches a method for optimal transparency processing in 
a graphical user interface to merge graphical images of objects, comprising a 
transparency control step of, when it is determined that an object is present, making a 
transparency of an image higher than that when it is determined that an object is not 
present (column 3, lines 40-46; column 6, lines 58-65). As was very well known to 
those of ordinary skill in the art, and illustrated by Buxton, image transparency enables 
a user to view multiple layers of information without losing information that would 
otherwise be occluded by overlapping images, thus increasing the efficiency and 
simplicity of the user interaction, as well as improving the quality of the imaging system 
as a whole. Therefore, it would have been obvious to a person having ordinary skill in 
the art at the time the invention was made to combine the transparent images of Buxton 
with the invention disclosed by Kato. 
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10. Claims 9 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Kato, in view of Stroyan (U.S. Patent No. 5,923,333), referred herein as Stroyan. 

Regarding claim 9, Kato teaches the method according to claim 1 , and further 
teaches the method, characterized in that in the virtual space image generation step, 
when the list image overlaps a hand region in the physical space image acquired in the 
physical space image acquisition step (column 12, lines 18-26). Kato does not explicitly 
teach the method, wherein the image of the virtual space is generated on the basis of 
priority data to designate which of the hand region and the list image should be 
rendered in front. Stroyan teaches a method for fast alpha transparency rendering, 
wherein an image of a virtual scene is generated on the basis of priority data to 
designate which of a plurality of virtual objects should be rendered in front (column 1 , 
lines 57-67; column 2, lines 1 -5). As was well known at the time of the invention, and 
illustrated by Stroyan, utilizing priority data to render objects in a virtual scene results in 
an accurate representation of the virtual scene, and accomplishes this in a fast and 
efficient manner. Therefore, it would have been obvious to a person having ordinary 
skill in the art at the time the invention was made to combine the priority rendering 
method of Stroyan with the invention disclosed by Kato. 

Regarding claim 10, Kato in view of Stroyan teaches the method according to 
claim 9, and further teaches the method, characterized by further comprising a 
designation step of designating which of the hand region and the list image should be 
rendered in front, wherein in the designation step, designated contents are set to the 
priority data (Stroyan, column 1, lines 57-67; column 2, lines 1-5). 
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Conclusion 

1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to DAVID T. WELCH whose telephone number is 
(571)270-5364. The examiner can normally be reached on Monday-Thursday, and 
alternate Fridays, 7:30am-5:00pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Xiao Wu can be reached on (571)272-7761 . The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/dtw/ 

/XIAO M. WU/ 

Supervisory Patent Examiner, Art Unit 2628 



